Mechanized gladhand

ABSTRACT

The present technology describes systems and methods for a mechanized gladhand and an end effector. The mechanized gladhand may have a collar capable of rotational motion while the mating surface may be in contact with a second surface without rotating. This may reduce reaction loads while maintaining the integrity of the connection. The end effector may include a drive motor, clasping tool, and/or driving installed tool that may be used to couple a tool with the gladhand or may be used in other applications. The end effector may have a compact design to allow precise control. The movement of the end effector with the gladhand may be controlled based on image processing to align the tool with another component.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims the benefit of U.S. Provisional Application No.63/035,089, filed Jun. 5, 2020, the complete disclosure of which ishereby incorporated herein by reference in its entirety.

BACKGROUND

Vehicles are shifting towards self-automation and self-driving modes. Inparticular, various aspects of tractor-trailer systems are also beingautomated. One such process is the connection of tractor-trailerpneumatic and electric lines. Often, each tractor is equipped with apneumatic hose and an electrical line to be connected prior to vehiclemovement.

It is with respect to these and other general considerations that theaspects disclosed herein have been made. Although relatively specificproblems may be discussed, it should be understood that the examplesshould not be limited to solving the specific problems identified in thebackground or elsewhere in this disclosure.

SUMMARY

The present technology relates to a mechanized gladhand and an endeffector. In an aspect, a mechanized gladhand is disclosed. Themechanized gladhand includes a sealing surface; a collar; a plunger; aturret rotatable about the collar and the plunger; and a retentionspring applying a threshold force on the plunger and the collar to limitrotation relative to each other.

In an example, the mechanized gladhand further includes a connectorplate; and a detent plate. In another example, the connector plate andthe detent plate are coupled to the turret to rotate with the turret. Ina further example, the turret is coupled to a hose and wherein the hoseis coupled to a truck. In yet another example, the hose is fluidlycoupled to a duct in the plunger and a port in the sealing surface. Instill a further example, the duct in the plunger and the port in thesealing surface are concentric. In another example, the collar includesa connector plate and a detent plate. In a further example, the plungeris retained to the turret on a rotational bearing.

In another aspect, a method for automating a gladhand coupling between avehicle and a trailer is disclosed. The method includes identifying, bya processor, a trailer mating surface, based on at least one image.Based on the at least one image, the method includes determining amating position of the trailer mating surface, based on the at least oneimage. The method also includes positioning an end effector based on themating position, such that a gladhand mating surface of a gladhandcoupled to the end effector is coupled to the trailer mating surface.Additionally, the method includes rotating the end effector and thegladhand relative to the trailer mating surface and the gladhand matingsurface, at the mating position. The method further includes decouplingthe gladhand from the end effector by opening a clamp of the endeffector; and repositioning the end effector.

In an example, the at least one image includes a first image and whereinpositioning the end effector is further based on a second image. Inanother example, the first image is obtained from a first camera and thesecond image is obtained from a second camera. In a further example, thesecond camera is coupled to end effector. In yet another example,positioning the end effector includes controlling at least one linearactuator coupled to the end effector. In still a further example,repositioning the end effector includes controlling at least one linearactuator coupled to the end effector. In another example, opening theclamp of the end effector includes moving a traveler along a drive shaftof the end effector.

In a further aspect, a method for automating a gladhand decouplingbetween a vehicle and a trailer is disclosed. The method includesidentifying, by a processor, a gladhand with a gladhand mating surfacecoupled to a trailer mating surface, based on at least one image. Basedon the at least one image, the method includes determining a matingposition of a gladhand coupled to the trailer mating surface. The methodalso includes positioning an end effector based on the mating position,such that the end effector becomes coupled to the gladhand.Additionally, the method includes rotating the end effector and thegladhand relative to the trailer mating surface and the gladhand matingsurface, at the mating position. The method further includesrepositioning the end effector and the gladhand.

In an example, determining the mating position is further based onmachine learning. In another example, the at least one image includes afirst image and wherein positioning the end effector is further based ona second image. In a further example, repositioning the end effector andgladhand includes coupling the gladhand to the vehicle. In yet anotherexample, rotating the end effector and the gladhand relative to thetrailer mating surface decouples the gladhand mating surface from thetrailer mating surface.

This Summary is provided to introduce a selection of concepts in asimplified form that are further described below in the DetailedDescription. This Summary is not intended to identify key features oressential features of the claimed subject matter, nor is it intended tobe used to limit the scope of the claimed subject matter. Additionalaspects, features, and/or advantages of examples will be set forth inpart in the description which follows and, in part, will be apparentfrom the description, or may be learned by practice of the disclosure.

BRIEF DESCRIPTION OF THE DRAWINGS

Non-limiting and non-exhaustive examples are described with reference tothe following figures.

FIG. 1 depicts a side view of a vehicle.

FIGS. 2A-2B depict example gladhand coupling attachments positioned on atrailer.

FIGS. 3A-3B depict example gladhand attachments positioned on a cab.

FIG. 4 depicts an example method for manual connection of gladhandsbetween a cab and a trailer.

FIGS. 5A-5B depict top-down illustrations of a tractor trailer system,including a workspace limited by dynamic operation of the tractortrailer system.

FIGS. 6A-6B depict an example location of a mechanized gladhand assemblyto be mounted and stored on a cab of a truck.

FIG. 7 depicts an example camera mount.

FIGS. 8A-8D depict linear actuators, and components of linear actuators,for moving an end effector in three-dimensional space.

FIGS. 9A-9D show various views of a customized, mechanized gladhand,otherwise referred to herein as a “mechand.”

FIGS. 10A-10J show an end effector, and components of an end effector.

FIG. 11 depicts a diagram of an example system architecture and control.

FIG. 12 shows an example setup of two cameras.

FIG. 13 shows an example setup of a light projector and a camera.

FIG. 14A shows an example point of view of a first camera.

FIG. 14B shows an example point of view of a second camera.

FIGS. 15A-15I show images that may be included in the identification andlocalization process performed on an image taken by a camera.

FIGS. 16A-16D show example image processing steps.

FIGS. 17A-17B show additional example image processing steps.

FIG. 18 shows an example camera setup.

FIGS. 19A-19G show example applications of one or more gabor filters onan image.

FIGS. 20A-20C show rectilinear projection and various forms of imagedistortion.

FIG. 21 illustrates an example of a suitable operating environment inwhich one or more of the present embodiments may be implemented.

FIG. 22 depicts an example method for automated gladhand coupling.

While examples of the disclosure are amenable to various modificationsand alternative forms, specific aspects have been shown by way ofexample in the drawings and are described in detail below. The intentionis not to limit the scope of the disclosure to the particular aspectsdescribed.

DETAILED DESCRIPTION

Vehicles are shifting towards self-automation and self-driving modes. Inparticular, various aspects of tractor-trailer systems are also beingautomated. One such process is the connection of tractor-trailerpneumatic and electric lines. Often, each tractor is equipped with apneumatic hose and an electrical line to be connected prior to vehiclemovement. Connecting these lines has previously been done manually bythe truck operator, and often requires a high level of dexterity andforce to hook up the lines. The ability to automatically connect air andelectric lines is a critical step in having an automated truck.

The process requires the airlines' connectors, gladhands, and theelectric line to be moved and secured to the front of the trucks'gladhand/electrical configuration. For a successful connection, theairlines are rotated and/or torqued into position and sealed to maintainoperational pressure (e.g., 140 psi), and the electrical plug is pushedinto its socket. Of the variety of connections, the present disclosurefocuses on the connection of the emergency pneumatic line (e.g., airline for the emergency braking system of the trailer), although thepresent disclosure may be applied to a variety of connections. Aproposed solution is detecting and locating the gladhands using a camerasystem and sealing the airline with a specially designed end effectorand modified gladhand. Although the present disclosure discussescomponents operating to connect lines of the vehicle to the trailer(vehicle-side), it should be appreciated that the components may beindependent of the vehicle and/or trailer, or may be operable toconnection the trailer to lines of the vehicle (trailer-side).

Two cameras may be used to identify and locate the gladhands. Thecameras may be mounted on the back of the cab. Custom software mayidentify and locate the gladhands from a depth map of the camera.

The custom gladhand (otherwise referred to herein as a “mechand”) may becapable of mating with any connector (e.g., including any connectorsunder the SAE J318 standard). The mechand may utilize an externallydriven rotational mechanism to mate with the trailer gladhand,minimizing the input torque and reducing the structural and actuatingrequirements of the end effector. The end effector may combine a clampand rotational drive mechanism to index, position, and actuate themechand.

Steel mounts may be fabricated and attached to the chassis. Additionallyor alternatively, mounts may be 3D printed or machined to hold the endeffector and computer vision components. A separate mount for the hoselines may be fabricated to imitate the setup on a real truck.

A mechanized gladhand and an end effector may be used to assist inautomating the gladhand coupling process. In aspects, the system mayidentify, locate, grab, and attach the emergency gladhand from the cabto the trailer. The system may use a computer as the main hubcontrolling each of the components of the system. Utilizing the camerasof the system, a custom computer vision suite may locate and identifythe gladhands on the back of the trailer and determine or calculate amating position for the end effector. A modified gladhand may be capableof mounting on the airlines, and removably couplable to the end effector(e.g., capable of being picked up by the end effector).

As described herein, a depth map may be an image with a channelassociated with recorded depths for each respective pixel. As alsodescribed herein, a gladhand may include connectors used to connect gasand power from truck to trailer. Additionally, a mechand, as usedherein, may be a customized or modified gladhand. An end effector maymoveable or positionable to interact with the environment. References toRealSense may be an Intel RealSense (RGB-D) depth camera D435. As usedherein, an “SDK” refers to a “software development kit,” a “POV” refersto “point of view,” and an “FOV” refers to “field of view.” As furtherused herein, a tractor and/or cab may mean the front end of atractor/trailer system.

FIG. 1 depicts a side view of a vehicle 100. In the example depicted,the vehicle 100 is a truck with a chassis supported by wheels 102. Thevehicle 100 may be a part of a tractor-trailer combination, ortractor-trailer system, which may include the vehicle 100 having aso-called fifth wheel by which a box-like, flat-bed, or tankersemi-trailer 108 (among other examples) may be attached for transportingcargo or the like. A distance D may be seen between the vehicle 100 anda trailer 108 when the vehicle 100 and the trailer 108 are aligned. Thedistance D varies, however, when the vehicle 100 and the trailer 108 arenot aligned, such as when the vehicle 100 is turning (sometimes referredto herein as “trailer swing”). Limited clearance 110 may exist betweenthe trailer 108 and the point of connection on the vehicle 100. Whilethe vehicle 100 is depicted as a truck in FIG. 1, it should beappreciated that the present technology is applicable to any type ofvehicle where gladhand connections are required or desired.

The example vehicle 100, otherwise referred to herein as a truck,tractor, or cab, includes a cabin 104 from which a driver may steer thevehicle 100. The vehicle may include a power and control system 106 tooperate the vehicle 100. The trailer 108 may include gladhand couplingassemblies to allow coupling of pneumatic lines with the trailer 108 toassist in releasing and applying the brakes on the trailer 108. Manualcoupling of the vehicle gladhands with the trailer may take asubstantially amount of a driver's time and/or may result in physicalexertion of the driver to obtain a tight connection. According to thepresent technology, a gladhand on the vehicle 100 may be altered tominimize physical exertion while obtaining a connection and/or thegladhand may be mechanized to couple with the trailer without userintervention (e.g., by using a linear actuator and/or an end effectordescribed herein). The components and operations of example gladhands,end effectors, linear actuators, and automated gladhand connection arediscussed in further detail, below.

FIGS. 2A-2B depict example connection ports 200 positioned on a front ofa trailer 226, as known in the art. The connection ports 200 may includetrailer emergency brake lines, connections that enable movement intandem, including a power supply, emergency pneumatic brake line, andnominal pneumatic brake line. The perspective view of example connectionports 200 shown in FIG. 2A includes a first gladhand connection site202, a second gladhand connection site 204, and a power connection site206. The power connection site 206 may include a movable protector 208for safety. The connection ports 200 may be mounted to the trailer 226with a bracket 210. Each port or site of the connection ports 200 iscoupled to the trailer 226. The first gladhand connection site 202 iscoupled to the trailer 226 via a first pneumatic line 212 and the secondgladhand connection site 204 is coupled to the trailer 226 via a secondpneumatic line 214. The power connection site 206 is electricallycoupled to the trailer 226 via a power line 216.

A side view of the first gladhand connection site 202 is shown in FIG.2B. A gladhand connection site, such as the first gladhand connectionsite 202, includes a sealing surface 218, a pneumatic port 220, aconnector plate 222, and a detent plate 224. The sealing surface 218frictionally seals the pneumatic port 220 when pressure is appliedbetween the sealing surface 218 of the connection site and a sealingsurface of a gladhand. The connector plate 222 of the gladhandconnection site is capable of receiving a detent plate of a gladhand andthe detent plate 224 of the gladhand connection site is capable of beingreceived by a connector plate of a gladhand. A gladhand is positionedand rotated at the gladhand connection site to removably couple thegladhand connection site with a gladhand. This process is sometimesreferred to as gladhand coupling.

FIG. 3A depicts an example gladhand 300 that is prior art. The gladhand300 includes a sealing surface 302, a pneumatic port 304, a connectorplate 306, a detent plate 308, and a pneumatic line 310 coupled to thevehicle (e.g., vehicle 100 in FIG. 1). As further described above withrespect to FIGS. 2A-2B, the gladhand 300 couples to a gladhandconnection site, such as gladhand connection sites 202, 204 shown inFIG. 2A. The sealing surface 302 frictionally seals the pneumatic port304 of the gladhand to the gladhand connection site when pressure isapplied between the sealing surface 302 of the gladhand and a sealingsurface of a gladhand connection site (e.g., sealing surface 218). Theconnector plate 306 of the gladhand is capable of receiving a detentplate of a gladhand connection site (e.g., detent plate 224) and thedetent plate 308 of the gladhand is capable of being received by aconnector plate of a gladhand connection site (e.g., connector plate222).

FIG. 3B shows an example tractor mount 312 for one or more gladhands(e.g., gladhand 300) coupled to a vehicle, as known in the art. Thetractor mount 312 may include one or more receiving portions 314 toreceive a gladhand 300 such that the pneumatic port 304 supplied by apneumatic line 310 from the vehicle is sealed. To seal the pneumaticport 304, the receiving portions 314 may include features of thegladhand connection site further described above.

FIG. 4 depicts an example method for manual connection of gladhandsbetween a cab (e.g., vehicle 100) and a trailer (e.g., trailer 108), asknown in the art. The gladhands follow strict guidelines outlined by theSociety of Automotive Engineers. The term ‘gladhand(s)’ may be usedinterchangeably with the term ‘connectors’ throughout this disclosure.

Connection of the pneumatic and electric lines from the cab to thetrailer has previously been done manually by a truck operator. Theseconnections often require substantial dexterity and force to be appliedby the truck operator to adequately couple the gladhands (e.g., form asecure, sealed connection between the gladhand and the gladhandconnection site). The method 400 shown in FIG. 4 shows an example ofmanual connection of a gladhand. The method 400 begins at operation 402where a gladhand is removed from storage mounted on the cab. The storageon the cab may include the tractor mount 312 described in FIG. 3B.Multiple gladhands may be stored and/or removed concurrently.

At operation 404, a gladhand configuration on the trailer is identified.In an example, the user performing the manual gladhand coupling mayvisually scan the front of the trailer to identify a trailerconfiguration (e.g., which may include a gladhand connection site) tocouple the gladhand removed from stored at operation 402. In someexamples, a gladhand may couple to a specific gladhand connection site.

At operation 406, a gladhand is manually aligned with the trailerconfiguration. Manual alignment of the gladhand may include aligning acomponent of the gladhand with a component of the trailer configuration.For example, a sealing surface of the gladhand may be aligned with asealing surface of the trailer configuration or a pneumatic port of thegladhand may be aligned with a pneumatic port of the trailerconfiguration.

At operation 408, the gladhand is manually rotated to be secured ontothe trailer at the trailer configuration. After being rotated, aconnector plate of the gladhand may be aligned with a detent plate ofthe trailer configuration and a detent plate of the gladhand may bealigned with a connector plate of the trailer configuration. Pressuremay be applied when rotating the gladhand, as required or desired. Thealignment of the sealing surfaces of the gladhand and the trailerconfiguration may result in frictional force, which may cause physicalexertion of the user providing the manual gladhand coupling.

FIGS. 5A-5B depict top-down illustrations of a tractor trailer system500, including a vehicle 502, a trailer 504, and a workspace 506 that islimited by dynamic operation of the tractor trailer system 500. Eachtractor trailer system 500 may have different parameters to consider,such as locations of mounted gladhands, hoses, and distance between thecab 502 and the trailer 504. Although aspects of this disclosure may bespecific to features of a Trailer 300 by Kenworth, it should beappreciated that aspects described herein may be applicable to a varietyof tractor trailer systems.

One consideration of the present disclosure is limited workspace 506.Placement of various components may be limited due to the dynamicmovements required by the cab 502 and trailer 504. The initial workspaceparameters may include a variety of example dimensions, including a 42″distance between the back of the cab 502 and the front of the trailer504 when the cab 502 and trailer 54 are aligned, and the entire width ofthe back of the cab 502.

The workspace 506 may be further reduced due to trailer swing shown inFIG. 5B. FIG. 5B shows a top-down illustration of a trailer swing duringa turn. Trailer swing occurs as the cab 502 makes a sharp turn. Duringtrailer swing, the back of the trailer sweeps an arc spanning a majorityof the space between the cab 502 and trailer 504. This may significantlyreduce the allotted working distance (e.g., from 42″ to 2″) at thecenter of the cab 502. The clearance between the trailer 504 and itschassis may be approximately zero when the trailer swings, because asystem installed above the rails may be destroyed. Further restrictionsto the workspace 506 may include locations of gas tanks on the vehicle,suspension systems of the vehicle, and other attachments to the vehicle.

The workspace 506 shown in FIG. 5B illustrates an unobstructed area(e.g., an area free from collisions) between the cab 502 and the trailer504 accounting for potential trailer swing. Thus, the workspace 506indicates a safe zone to mount components of the described technologythat is not impacted by trailer swing, constriction by a gas tank, andthe suspension system of the vehicle. To the side of the chassis, amodular space may be designated for external generators, step stools,and other components. In an example, this modular space was measured tobe 36″×32″×30″ in volume.

FIGS. 6A-6B depict an example assembly location 608 of mechandcomponents in a workspace 606 on the back of a cab 602 of a vehiclesystem 600. Because trailer swing restricts the workspace 506 near thecenter of the cab 602, the assembly location 608 may be desired to bepositioned towards a side of the cab 602. As shown in FIG. 6A, theassembly location may be near the driver's side of the cab 602 for easyaccess by a truck operator.

The mechand components to be included in the assembly location 608 mayinclude an end effector, control system, and computer vision. The designmay account for placement of the system on the truck 602, supportstructures for mounting, and the motions of the system. The computervision system may include a primary system and a secondary system. Theprimary system may utilize a first, depth camera, mounted to the truck602, capable of identifying and locating the gladhand connection site(s)on the back of the trailer. In an example, the first camera may be anIntel RealSense®. After the gladhand connection site(s) are located bythe primary system, this information may be used to position orreposition the end effector to a location near a gladhand connectionsite. The secondary camera system may be mounted on the end effector andmay function by locating the pneumatic port (e.g., center hole) of thetrailer-mounted gladhand connection site. The control system may performor send instructions to another component to move or position the endeffector to precisely line up the end effector for the gladhand couplingprocedure. The end effector may rotate, along with rotating a coupled,modified gladhand, such that the modified gladhand is in a matingposition, sealing the connection and pressurizing the trailer.

FIG. 7 depicts an example camera mount 700. The camera mount 700 iscapable of securing a camera. For example, a camera may rest on asupport surface 702 of the camera mount 700. The camera may be removablycoupled to the camera mount 700 via clamping or tightening onto thesupport surface 702, such as at the securing gap 706 in the supportsurface 702. The camera mount 700 may be mounted on the back side of thecab. For example, the camera mount 700 may be secured via screws orsimilar means via access holes 704. The primary camera system (e.g.,Intel RealSense) may be mounted onto the back of the cab looking towardsthe trailer. The camera mount may be 3D printed, molded, or otherwisefabricated. The camera mount 700 shown in FIG. 7 shows one example of acamera mount and other mounts or supports are appreciated. For example,the camera mount 700 shown in FIG. 7 may only be a portion of a complexcamera mount.

FIGS. 8A-8D depict linear actuator assemblies 800A, 800B, and componentsof linear actuator assemblies, for moving an end effector inthree-dimensional space. The linear actuator assemblies 800A, 800B maybe positioned between the cab 808 and the trailer 810 in a workspacefree from potential collisions, such as the position shown in FIGS.8C-8D. As shown, the linear actuator assembly 800A, 800B is positionedon the chassis of the cab 808 below the base 812 of the body of thetrailer 810.

Linear actuators may run on electricity (e.g., 24V DC) with feedbackcontrol compatibility. For example, as shown in FIGS. 8A-8B, a liftingcolumn 802 may include a telescopic lead screw. In another example, thelifting column 802 may include end-of-stroke limit switches. In afurther example, the motor may be connected via a cable. In anotherexample, the lifting column 802 may include encoder position feedback.In an example, the lifting column 802 may be an LC2000. In anotherexample, the lifting column 802 may have a variety of specifications,including a maximum load, a maximum load torque, available inputvoltages, minimum ordering stroke, maximum ordering stroke, and leadcross section. In an example, the lifting column 802 has the followingspecifications: a maximum load of 2000 N, a maximum load torque 150/500Nm (dynamic/static), speed of 19/15 mm/s (no load, maximum load),available input voltage of 24 VDC, minimum ordering stroke of 200 mm,maximum ordering stroke of 600 mm, and lead cross section of 1.5 mm².

A lifting column 802 is shown in FIGS. 8A-8B for vertical motion. Thelifting column may be a robust telescoping linear actuator with aplatform on the top and a retracted length of, in examples,approximately 17 inches. A platform 806 a, 806 b on the top of thelinear actuator assembly 800A, 800B may be capable of handling largemoments which would be applied to the platform as the end effector moveson the horizontal actuators providing a downward force. In an example,the lifting column 802 is an LC2000 series lifting column from ThomsonLinear compatible with feedback controls and capable of being integratedinto the control system.

Horizontal rails 804 (otherwise referred to as linear actuators 804)and/or actuators may be used for horizontal motion. In an example, threelinear actuators 804 may be used to move linearly in an XYZ coordinatesystem. For example, the linear actuators 804 may allow linear movementin two directions: x-direction (side-to-side motion) and y-direction(forward-backward motion). The third direction (vertical direction), thez-direction (up-down motion), is controlled by the lifting column 802,which supports the base plate 806 holding the linear actuators 804. Thehorizontal (x- and y-direction) motions may be enabled by a two-axisgantry system including two rodless linear actuators 804 and a linearactuator 804 mounted on a base plate 806. In an example, the linearactuators may be belt driven, rodless linear actuators due to theirdurability and low maintenance requirements, even in dirty environments.In an example, the actuators may be SIMO Series rodless, linearactuators manufactured by PBC Linear. The linear actuators may be usedto position the end effector.

FIGS. 9A-9D show various views of a customized, mechanized gladhand,otherwise referred to herein as a “mechand,” “modified gladhand,” or“mechanized gladhand.” The changes shown from the features of a standardgladhand result in an easier autonomous mating process. The connectionbetween two standard gladhands is traditionally made by aligning thecontact flanges at an approximately ninety-degree offset, contacting thecentral gaskets (e.g., sealing surfaces), and then rotating the flangesinto position (e.g., the gaskets of each gladhand rotate relative toeach other, with their respective flanges, to be mated). The frictionopposing this process can be substantial and is a product of the totalspring force (provided by the rubber gaskets) required to hold thegladhands in a locked position. Due to substantial frictional forces,gladhand coupling with a traditional gladhand can be difficult toperform manually. The customized gladhand (“mechand”) shown in FIGS.9A-9D is a rotationally articulated connection system that may interfacedirectly with the standard gladhand connection site geometry, includingas specified in SAE J318. In examples, the mechand shown reduces thetorque required in the connection process by a large factor (e.g.,approximately ten-fold); this in turn enables a significant reduction inthe weight, cost, and complexity of an end effector to be used tofacilitate an automated gladhand coupling, and a proportional reductionin the capability of the gross movement system.

The mechand shown in FIGS. 9A-9D reduces the force required byarticulating the connecting flanges in relation to the central plungerout of contact with the flanges on the gladhand, and then extending thecentral plunger to bring the flanges into contact and provide therequired clamping force. The design allows these steps to be completedsequentially, with no control input other than a unidirectionalapplication of mechanical torque to an externally accessible drive axle.

The mechand may include a variety of rotational groups. For example, themechand may include three main rotational groups: the plunger 912, thecollar 918, and the turret 906. The outermost section (turret 906)includes the main body of the mechand 900 and includes the mountingpoints for the air hose and drive pinion bushing. The innermost section912 or plunger 912 may be retained to the turret 906 on a rotationalbearing. The plunger 912 may include a drive gear that mates with thepinion on the turret 906 and a central face-cam that impinges on thefinal section 918 or collar 918. The collar 918 may be locatedconcentric to the axis of the turret 906 and the plunger 912; may beconstrained against the rear of the plunger face by a face-cam(complimentary to that on the plunger) and/or against the turret 906 bya spring 916. The collar may include imitations of the flanges on thestock gladhand and may not be rotationally constrained to the turret906, and thus may be free to rotate within the constraints imposed bythe interaction of the plunger 912 and collar cam 914. Thespring-loading of the collar against the turret 906 (and the resultingaxial force against the rear of the plunger) may ensure constantengagement of the cam mechanism.

During the mechand attachment and detachment operation, the turret 906may provide the geometry on which the end effector clamp engages anddisengages, and/or may provide index points for the mating of the endeffector drive shaft and mechand drive pinion. During normal operation(e.g., the mechand attached to gladhand and the tractor-trailer is inmotion) the turret 906 can rotate freely against the collar and plunger,providing an element of mechanical flexibility to the hose connectionand reducing the external force on the clamping system. An o-ring may beincluded between the joint of the turret 906 and the plunger 912 toprovide a sealed duct through which air can flow from the mechand hoseto the trailer brake circuit.

During attachment, detachment, and normal operation, the plunger mayprovide the contact area between the central rubber gasket on the stockgladhand and the mechand. The plunger 912 may have a central air-ductfor the hose circuit, and concentric and external to that duct may bethe plunger face cam. The normal vector of the face cam may be oppositethat of the plunger contact with the gladhand. Axially aligned to theduct axis and external to the plunger cut, the collar may provide thestructure for a face cam (complimentary to that of the plunger) and themounting of the structural flanges.

The face-cams on the plunger and/or the collar may provide themechanical interaction for the function of the mechand. When the collaris unconstrained, the force of the turret-collar-spring may prevent thecams from rotating against each other. When the flanges of the collarsare impinged by the flanges on the gladhand, the force between the camsmay overcome the axial force of the spring (e.g., a threshold force),and may induce a relative rotation between the cams and force the collaraway from the gladhand, which may bring the mechand flanges into contactwith those on the gladhand.

FIG. 9D shows an exploded view of the mechand. The mechand may include aplunger 912, a collar 918, a plunger cam 924, a collar cam 914, a turret906, a turret ring 920, a turret bevel gear 922, a turret pinion gear926, a turret pinion axle 928, a collar retention spring 916, a lugplate 908, a plunger bellow retention insert 930, a bellow retentionring 932, a plunger/collar bellow 902, a collar sleeve 904, and a collarweatherseal 934.

The mechand may be designed to reduce reaction loads on the trailercoupling process. Additionally or alternatively, the mechand mayincrease the potential to automate the tractor-trailer coupling processby first redesigning a gladhand (an interlocking hose coupling device)fitted to hoses supplying pressurized air between the host (chassis) andthe accessory (trailer). The mechand may improve the connectionreliability and reduce the required forces during the handshake from theconnection process. The two mating surfaces may be continued, but maynot contract or rotate, while only the collar has the rotational motionto perform the locking mechanism of the handshake. Once the rotationalmotion of the collar has completed, a mechanical system behind themating surface may generate the sealing force. Alternatively, thesealing force may be generated prior to any rotational motion, or duringrotational motion of the flange. This new process may drastically reduceinstallation reaction loads due to the elimination of the frictionforces between the two mating surfaces. This reduction in reaction loadsmay permit automating the process with a lightweight extending arm whilemaintaining the integrity of the connection.

The mechand may have a variety of advantages, including a combinedcenter-locating and automatic shut off, lower coupling forcestransmitted back through the gladhands, adaptable to mate with multipledrivers other than gladhands, useful with multiple tools by the endeffector described herein, compact in size of the gladhand package,and/or increased difficulty for inadvertent disassembly for increasedsecurity. Additional advantages may include alleviating the rotatingmotion on the mating surfaces around the central air path, combining thecentral locating feature and the air control mechanism, reducingreaction forces during the coupling process, simplifying automationprocesses by reducing assembly reaction forces, combining locatingfeature and actuation and by mechanizing the ability to increase surfacepressure forces for seal integrity, or allowing multiple automatedcontrol features to locate and clamp to mating surfaces.

FIGS. 10A-10J show an example end effector and components of the exampleend effector. FIG. The overall assembly of the end effector is shown inFIG. 10A. As shown in FIGS. 10B-10C, the end effector has at least twoconfigurations: a clamped configuration 1000A of FIGS. 10A, 10B, and10J, and an unclamped configuration 1000B of FIG. 10C. In the clampedconfiguration 1000A, the top clamp 1002 and bottom clamp 1004 of the endeffector are positioned such that a gladhand is securable between thetop clamp 1002 and the bottom clamp 1004. In the unclamped configuration1000B, the top clamp 1002 and bottom clamp 1004 are angled and spacedapart such that a gladhand is not securable. FIG. 10J shows the endeffector in a clamped configuration 1000A securing the mechand 900 ofFIGS. 9A-9B. FIGS. 10E and 10F show the shifter carriage 1014 of the endeffector in the clamped configuration 1000A and unclamped configuration1000B, respectively. In the clamped configuration 1000A, the traveler1012 is away from the fork. In the unclamped configuration 1000B, thetraveler 1012 is near the fork.

The end effector may include a top clamp 1002, a bottom clamp 1004, atop clamp arm 1006, a bottom clamp arm 1008, a ball-detent clutch 1010,a traveler 1012, a shifter carriage 1014, a drive motor 1016, a gear box1018, a camera mount 1020, a camera 1022, a friction plate 1024, apneumatic actuator 1026, a drive gear 1028, and a clamp carriage 1030.The drive motor 1016 provides power to transition between the clampedconfiguration 1000A and the unclamped configuration 1000B of the endeffector. The ball-detent clutch 1010 engages or disengages a drive gear1028 coupled to the gear box 1018 to cause or stop movement ofcomponents of the end effector. When the ball-detent clutch 1010 isengaged, the gear box 1018 and the drive gear 1028, via the pneumaticactuator 1026, causes rotation of a ball-end hex drive shaft 1032 andmoves the traveler 1012 about the length L of the end effector along theball-end hex drive shaft 1032. When the traveler 1012 is outside of theshifter carriage 1014, the end effector is in the clamped configuration1000A. Based on the position of the traveler, force is applied to thefriction plate 1024 to cause movement of at least the top clamp 1002 andthe bottom clamp 1004, relative to each other and the clamp carriage1030. When at least a portion of the traveler 1012 is inside the shiftercarriage 1014, the end effector is in the unclamped configuration 1000B.The camera mount 1020 is configured to support a camera 1022approximately parallel to the length of the top clamp 1002 and thebottom clamp 1004 in a position in which a view of the camera includesthe top clamp 1002 and the bottom clamp 1004. In this position of thecamera 1022, a view is provided that includes the top clamp 1002 and thebottom clamp 1004, relative to a mechand to be coupled to the endeffector 1000A, 1000B.

A static fork mounted to the body of the end effector enables the endeffector to angle the top clamp 1002, top clamp arm 1006, bottom clamp1004, and bottom clamp arm 1008 into the clamped configuration 1000A andunclamped configuration 1000B to engage and disengage with a body of amechand. As part of the end effector clamp engagement, a ball-end hexdrive shaft 1032 may be extended to mate with the mechand axle. Aclamping force may be produced between the top clamp 1002 and the bottomclamp 1004 by a mechanism of the end effector.

The end effector may include a camera mount 1020 for a camera 1022(e.g., a Logitech webcam) and may drive the mechand to complete gladhandmating process. The end effector may be the link between the mechandconnector and the gross movement system. The end effector may include avariety of functions. For example, the end effector may mechanicallyindex with the mechand body when unconnected in a manner that correctsfor a given positional error of the gross movement system. Additionallyor alternatively, the end effector may apply torque to the mechand driveaxle through a drive shaft. This shaft reliably engages as part of themechand/end effector gripping process. In another example, the endeffector may provide a stable platform for translational movement of themechand and mechanical indexing on the geometry of the trailer gladhand.

The end effector may allow a compact reactive system that leverages aclutched torque limiting clamp system. The end effector may include acompact drive motor 1016 with multiple limit switches to place themechanized gladhand into position. The end effector may be used toproperly align and couple the gladhand to its mating half and completethe hose interlocking process. The end effector may also perform thereversed process (e.g., de-couple the gladhand from a second matingsurface). The end effector may include a clutching system that allowsthe engagement of the tool. The end effector may have a single, compactmotor that may be used for multiple applications, including gripping andactivating the tool (e.g., gladhand). The end effector may have a forkand clamp design to secure and/or couple to the hose. The end effectormay include a precise force and movement control with limit switches.

Advantages of the end effector may include using the drive motor 1016,clasping tool, and/or driving installed tool in multiple applications.Additionally or alternatively, the end effector may include a mechanicalclutch 1010 for properly engaging the tool. The end effector may have acompact design with precise control.

As described herein, the end effector may include a drive motor 1016.The drive motor 1016 may be an electric motor, pneumatic motor,hydraulic motor, or any other type of motor. In an example, the endeffector may include a single electric motor. The motor may have a smallinline gear box 1018 and a shifter carriage 1014 to direct power toeither the drive (e.g., via the drive gear 1028) and/or clampingmechanisms. Control input may be minimal; motor power may be on/off witha single rotary encoder, and the state of the system may be trackedthrough a series of limit switches. The end effector system may becompact enough to be mounted on a short-travel tilt axis, allowingcompensation for non-vertical trailer gladhands. The entire system maymount with a four-bolt pattern on a single plane, providing flexibilityfor mounting to a future gross movement system.

FIG. 11 depicts a diagram of an example system 1100 for architecture andcontrol. In an example, the system architecture includes a computer 1102that may perform computer vision processing and/or a main script for thecontrol system 1100. As shown in this example system 1100, two camerasmay be used to take images and provide image data to the computer 1102for processing. Based on the image processing, instructions and/orinformation is sent to a controller 1108 to control the linear actuator1110 (e.g., further described with respect to FIGS. 8A-8D) and sent to acontroller 1112 to control the end effector 1114.

In an example, the controller 1108 to control the linear actuator 1110is a Raspberry Pi® 3 Model B, which controls stepper motor drivers forthe linear actuator using a Windows® PC running Python libraries PIGPIOand PIZERO. This example allows for integration of the computer 1102with the computer vision processing code with control of the linearactuator 1110, including positional feedback on forward and backwardmotion from the computer 1102 implementing the vision system. In anotherexample, the controller 1112 may be an Arduino® UNO. The controller 1112may include a microcontroller (e.g., ATmega328P), digital I/O pins(e.g., 14 pins), and/or analog input pins (e.g., 6 pins). The controller1112 may control an end effector (e.g., the end effector described inFIGS. 10A-10J) to position and rotate a gladhand or mechand (e.g.,described in FIGS. 9A-E). The controller 1112 may use data provided by acomputer 1102 and other sensors to perform the necessary movements tomake a gladhand coupling. A direct current (DC) motor on the endeffector may be controlled using a quadrature encoder and several limitswitches to provide responsive positional feedback for the system.

An example procedure for utilizing a single linear actuator to performthe gladhand coupling process may include the following steps: (1) airsupply activated (100 PSI); (2) electric supply powered on (12V, 10 A);(3) primary computer vision system takes a snapshot that generates a 3Dimage used to identify and locate the gladhands; (4) primary computervision system then locates the end effector, if unsuccessful, the endeffector moves forward slightly and the process is repeated untilsuccessful; (5) the position of gladhands are sent to a processor, whichsignals a linear actuator to move the end effector; (6) secondary camerasystem is activated and takes a snapshot locating the center of thegladhand's gasket; (7) the processor sends a signal to a linear actuatorto move the end effector to the required location as provided by thesecondary camera system mounted to the end effector; (8) the computersignals the end effector controller to start the sealing process,engaging the end effector; (9) when the connection is complete, the endeffector controller returns an all clear signal; and (10) the actuatorcontroller returns the end effector to a retracted position away fromthe mounted gladhand by adjusting the linear actuator.

FIG. 12 shows an example camera setup 1200 of two cameras. The camerasetup 1200 shows a stereo vision setup for a pinhole camera. Stereovision is a technique that generates 3D depth information using a pairof cameras 1202, 1204 with similar properties mounted a knowntransformation apart from one another. The two cameras may capture thesame scene from different perspectives at the same time. For a basicpinhole camera, a point in space may be projected onto the camera'simage plane by going through a camera's optical center. The possibledepth of a pixel may be anywhere on the projection line from the pointto a camera's optical center. To figure out the appropriate depth everypixel from one camera may be matched to image pixels in the othercamera. Then theoretical lines of projection may be made originatingfrom each camera's optical center that goes through the matched point.The intersection P of these projected lines is the location of thecoordinate in space. Then, using the transformation between the twocameras, three side lengths of a triangle formed between theintersection P and the two cameras may be used to calculate an object'sdepth.

FIG. 13 shows an example setup 1300 of a light projector 1302 and acamera 1304. The setup 1300 shown in FIG. 13 is an alternative to thesetup described in FIG. 12 to derive a depth of an object. Similar to astereo vision setup, a structured light setup 1300 uses two sensors. Oneof the sensors is offset at a known distance from the other. Instead ofusing two cameras, however, this setup 1300 uses a light projector 1302emitting a known light pattern and one camera 1304. By projecting aknown light pattern onto a surface of an object 1306 and capturing thatpattern with the camera 1304, a shape of an object's surface may bedetermined. If the light pattern detected by the camera 1304 is the sameas the light pattern emitted from the light projector 1302, then thesurface of the object is flat. If, however, the surface is not flat,then the light pattern may become distorted when captured by the camera.

FIG. 14A shows an example point of view (“POV”) 1400A of a first cameraand FIG. 14B shows an example point of view 1400B of a second camera. Inan example, the first POV 1400A is captured by a camera mounted on theback of a cab (e.g., with camera mount 700) and the second POV 1400B iscaptured by a camera mounted on the end effector (e.g., with cameramount 1020).

Referring to FIG. 14A, the first POV 1400A (e.g., of a primary camerasystem) shows an emergency gladhand 1402, an electrical box 1404, and anominal gladhand 1406. FIG. 14B shows the POV from a secondary camerasubstantially parallel to the primary camera and showing the emergencygladhand 1402.

The stereo view of the two cameras, as further described with respect toFIG. 12 may be used with the vision control system of FIG. 11 toidentify and localize the emergency gladhand 1402 to position the endeffector. Identification of the gladhand(s) includes determining anoutline of the Gladhand(s). Localization determines the position orcoordinates of the gladhand(s). The computer implementing imageprocessing may be capable of performing image analysis via machinelearning and/or via datasets of gladhand images. In an example, computervision may process, segment, identify, and localize the gladhand from adepth map and colored images produced by a camera.

The localization algorithm of the gladhand geometry may be broken downinto steps. For example, the steps may include (1) camera calibration,(2) pre-processing: gabor texture filter, (3) search forelliptical/circular objects, (4) depth calculation (estimation), and (5)deprojection of pixel to world coordinates. The camera calibration stepmay be used to gather coefficients relating to the camera's intrinsicand extrinsic properties. Factors of intrinsic and extrinsic propertiesinclude distortion coefficients, principal point, and camera focallength.

At least one of the cameras may be a camera capable of creating a 3Drendering of a scene in an outdoor environment. In an example, thecamera may be an Intel® RealSense® D435. In another example, the cameramay be accurate up to 10 meters. The camera may have a variety oftechnical specifications. For example, the camera may include a depthsensing range, an RGB resolution and framerate, a depth resolution, adepth field of view, an RGB field of view, physical cameral dimensions,and a connector. In an example, the camera may have the followingspecifications: a depth sensing range of 0.1-10 m, an RGB resolution andframerate of 1920×1080 FPS, a depth resolution of 1280×720, a depthfield of view of 87°±3°×58°±1°×95°±3°, an RGB field of view of(69.4°×42.5°×77°±3°, physical cameral dimensions of 90 mm×25 mm×25 mm,and a USB-C 3.1 connector. In another example, the camera may utilizestereo vision technology and structured light to calculate depth withinan image of the camera. In a further example, the camera may be equippedwith an IR projector, two stereo modules, and an RGB camera sensor. Thecamera may be capable of performing in both indoor and outdoorenvironments up to a range of 10 m. The primary camera system may useIntel's RealSense D435 depth camera. Utilizing a combination ofstructured light and stereo vision techniques a 3D image can begenerated from the scene. In the example provided above, the RealSense®projects an infrared pattern onto objects and measures the disparitybetween the two depth sensors to generate a depth map/image. The imagesproduced can then be transformed and overlayed onto images captured fromthe RGB sensor. A more detailed description of stereo vision andstructured light is included above in discussions of FIGS. 12 and 13.

An example of a secondary camera is a Logitech® Webcam C930E. Thesecondary camera system may act as a terminal guidance sensor. Thesecondary camera may examine the gladhand at a much closer range thanwhat is capable for a 3D camera, or depth sensor on the primary camera.As an alternative, other cameras may be used. As another alternative,software may locate the contact point on the gladhand.

Camera selection may include several factors, such as sensing max/minrange, cost, flexibility, outdoor durability, and developer support. Thepresent technology may use a depth map from a distance between 30″-42″,which would span a truck's workspace.

FIGS. 15A-15I show images which may be included in the identificationand localization process performed on an image taken by a camera. Animage may be 2D or 3D. FIG. 15A shows an original color image. FIG. 15Bshows a raw depth map of the original color image. FIG. 15C shows aback/foreground of the original color image. FIG. 15D shows a mode depthremoval of the original color image. FIG. 15E shows an iterative pixelremoval of the original color image. FIG. 15F shows a morphologicalfilter of the original color image. FIG. 15G shows an electrical boxidentification in the original color image. FIG. 15H shows a contourclassification of the original color image. FIG. 15I shows a pointestimation identification of the original color image. An identificationalgorithm can be broken down into three main steps, including apre-processing of depth map to remove noisy data; a pixel iteration andmorphological filtering to find the foreground; and a contourclassification to differentiate blobs and shapes.

A 3D image generated by a camera may be distilled down to the contoursof the gladhands, electrical box, and other miscellaneous componentsremain in the image. The contours may be classified based on their imageproperties and physical parameters, to differentiate between theemergency gladhand and other components (e.g., the electrical box).

The depth information collected may not be accurate. For example, thelarge black swaths and speckled black holes in FIG. 15B represent nulldata. Null data could be caused by sharp changes in depth, occlusion ofone or more depth sensors, varying ambient lighting interfering withcamera exposure, or misrecognition of the projector pattern. Inaddition, smooth surfaces also may have slight variations in their depthreadings and may require further manipulation.

In an example, processing the images from a 3D camera may include one ormore assumptions. For example, an assumption may include a shape of thesurface on which the gladhand is mounted. For example, the gladhand maybe mounted to a flat surfaces (e.g., a trailer that is flat andrectangular as viewed from back of the cab). This assumption relates towhether or not the trailer has a uniform depth relative to the camera.As another example, an assumption may be related to a mounting locationof the camera. For instance, an assumption may include that the camera'smounting location has been chosen so the field of view is composed ofmainly the back of the trailer. In this example, the back of the trailermay be the largest object within the camera view. In an example, theremay be an assumption that the trailer is at a uniform depth the depthmeasurement occurring most often will be that of the trailer, such thatany depth measurement with this value or farther away can then beignored. As a further example, an assumption may be related to how farthe gladhand protrudes from the trailer. For example, the gladhand mayprotrude about 4″ from the back of the trailer, plus or minus the heightof any additional plate it could be mounted on. In an example where theworking distance is at 42″, any object closer than 36″ is unlikely to bethe gladhand and may also be removed. Another assumption anyconfiguration of vehicle, trailer, and gladhand may be considered. Forexample, gladhands of different shapes, locations, or dimensions may beconsidered when making assumptions. Additionally, assumptions may berelated to the type of trailer, such as a box trailer, tanker, flat bedtrailer, or any other trailer type.

After the above steps have been performed, the image may appear similarto FIG. 15D. Further processing may be performed. The remaining objectsare of known sizes and take up a consistent amount of pixels each time.The maximum depth in the image is then reduced in small increments(e.g., 1 mm) until only a certain number of pixels remain. What remainsare objects closest to the camera, such as what is shown in FIG. 15E,including the gladhands, electrical box, and hoselines, along with somenoise.

Due to close proximity of the objects, some of the contours areconjoined as singular entities. To separate the contours into individualunique objects, morphological filters may be applied to the remainingentities. Morphological filters work to reduce noise in an image bymanipulating a binary image based on its shape. A variety ofmorphological operations may be used, for example, morphologicaloperators may include erosion and dilation. Where foreground objects arerepresented as white and background objects are black.

The erosion operation removes pixels at the boundary of foregroundobjects tightening up objects in the image and also works to remove anylone specks. Erosion also helps differentiate objects in closeproximity, by separating closely connected contours, such as the hosesconnected to the gladhands.

Dilation may follow erosion. Shrinking the foreground objects may removesome noise in the image and help separate objects of interest, but italso alters their physical size. To properly identify the gladhand, theimage needs to be as close as possible to its actual proportions. Thedilation operator counteracts this by expanding the foreground objects.The noise removed from erosion may not reappear in the image and theobjects may still be kept separate. What remains are blobs of theforeground objects, such as those shown in FIG. 15F.

The next few steps focus on differentiating between the contours. In anexample, a variety of components may be identified. For example, in FIG.15G, the emergency gladhand (e.g., the gladhand on the right side of theimage), the left-hand gladhand, and the electrical box are eachidentified. In FIG. 15G, the electrical box can be readily identifieddue to its hexagonal contour. Searching over all the contours theperimeter, the area of each contour is recorded. Using the equation fora hexagon, a theoretical area can be calculated and compared to the onerecorded in the image. This equation may be limited to use withhexagonal shapes (e.g., the electrical box) for the trailer.

To classify the contours, a variety of approaches may be considered. Forexample, two approaches include (1) Hu Image Moments, and (2) ElectricalBox Detection. In the Hu Image Moments approach, the classificationmethod may be based on the gladhand configuration. The camera may bemounted on the back of the cab and capture multiple sets of images ofgladhand configurations. The images may undergo image processing stepsoutlined herein. Features called Hu Image Moments may then be extractedand recorded for each contour.

Hu Moments are seven numbers that may include scale, rotation,translation, and reflection invariant, meaning if the image is shifted,rotated, enlarged, or flipped it may still have the same Hu Moments.Using a set of around 180 images, the Hu Moments, area, perimeter ofeach foreground object may be calculated. Using the mean and standarddeviation for each value, shape moments may be compared.

Additionally or alternatively to the Hu Moments, the electrical boxmethod may be used. For example, the electrical box method may be usedselectively, as a determining factor, where Hu Moments are similarbetween different shapes. In an example where the gladhands are roughlyat the same height of the electrical box, the centroids of each contourmay be checked to see if they had roughly the same y value. In thatexample, the Hu Moments may be matched to see if the right-hand gladhandwas found.

FIGS. 16A-16D show example image processing steps. FIG. 16A shows anoriginal color image, FIG. 16B shows the original color image height andwidth to determine an aspect ratio, FIG. 16C shows the gladhand contour,and FIG. 16D shows point estimation with a bounding box. The identifiedemergency gladhand contour is then be localized. In FIG. 17A, thesealing surface of is identified, as indicated by the dashed line 1702.Although a dashed line is shown in FIG. 17A, an indicator may beprovided, such as a green circle around the identified sealing surface.An approximate contact point may then be mapped from 2D imagecoordinates to 3D world coordinates.

From the identified contour (e.g., the sealing surface), the gladhandshave a specific geometry that may directly translate over to the image.For a sufficient contour, a center 1704 of the contour is alsoidentified. The objects' aspect ratio, the ratio of the width of theobject over its height, may be used to approximate a point that may beprojected where the secondary camera system may be repositioned.

FIGS. 17A-17B show additional example image processing steps. FIG. 17Ashows a center dot 1704, in the circle identifying the sealing surface1702, to represent the point needed to be mapped from 2D imagecoordinates to 3D world coordinates relative to the camera. FIG. 17Bshows dots 1706 scattered about the identified gladhand 1700 toillustrate possible 3D points calculated from the primary camera systemfor a possible location of the contact point. The primary camera mayfind an approximate location of the contact point (e.g., center dot 1704in FIG. 17A), and may forward its position to the computer. An exampleof possible points the primary system calculates are represented by thedots 1706 in FIG. 17B. The computer may calculate how far away thecontact point is relative to the camera's center and may maps the 2Dimage coordinates to 3D world coordinates. The coordinates may beprovided to the computer via a feedback loop, and the belt drive maymake adjustments to align the end effector.

FIG. 18 shows an example camera setup 1800. As shown, the setup 1800shows a camera 1802 with a field of view 1804 de-projecting image pointsto real-world 3D points P(X,Y,Z). The camera may be capable ofcalculating the depth of an object in space, however, the camera mayalso be capable of calculating (X,Y) coordinates as well. For example, acenter 1704 identified in FIG. 17A can be deprojected from imagecoordinates to real-world coordinates using the camera's known field ofview and the depth of the gladhand.

Using geometry, if an accurate depth measurement is known, then anobject's' height may be calculated using the vertical field of view(“FOV”) from the technical specifications from the camera. The belowequation may be used to calculate a height of an object for a given FOVand depth:

${{{Object}\mspace{14mu}{Height}} = {{\tan\left( \frac{FOV}{2} \right)}*{Depth}}},{{FOV} = \theta}$

In an example, the object is the entire image frame and it is assumedthe object is a flat wall at a specified depth. Using the aboveequation, the length and width of the entire image frame may becalculated in real-world units. For a given image resolution (e.g., 640,480), a ratio can be generated of length per pixel in a particularorientation. For example, at a set distance of 0.9 m and a FOV of 45degrees, the height of the image frame is 0.37 m. If the imageresolution is (640, 480) then the ratio of length per pixel is 0.37m/480 px or every pixel has a height of 7.7e⁻⁴.

Using this length per pixel ratio, every pixel's location as a point inspace can be measured relative to the camera's optical center, assumedto be one half of its image resolution. The camera's optical center isalso de-projected to be in real-world coordinates. The result is a pointin space measured as a distance D from the camera 1802 to the object1806.

The image pixel representing the point of contact can be projected fromimage plane to real-world coordinates relative to the camera. The depthof the object (in pixels) is used to complete the 2D to 3D mapping. Apixel from a non stereo-vision camera may be calculated based on ascaling factor known from objects within the scene. The gladhand'sgasket diameter may be measured. For example, the gladhand's gasket mayhave a diameter of roughly 39 mm. Using the major axis calculated in theprevious step as the object's height in units of pixels and the camera'sfocal length in pixels the depth can be calculated using the followingequation:

${Depth} = {{{Object}({mm})}*\frac{{focal}\mspace{14mu}{{length}({px})}}{{object}\mspace{14mu}{{height}({px})}}}$

The following equations can be used to calculate the 3D real-worldcoordinates of an object for a pinhole camera, assuming there is no lensdistortion, where X,Y,Z are 3D real-world coordinates, u and v are imagecoordinates, and fx,fy is the image focal length, cx,cy is the principalpoint from the camera matrix, and Z is the depth.

${X = {\frac{Z}{f_{x}}\left( {u - c_{x}} \right)}}{y = {\frac{Z}{f_{y}}\left( {v - c_{y}} \right)}}$

The image distortion may be taken into account to calculate a moreprecise measurement for the object's real-world coordinates. FIGS.20A-20C include further discussion about distortion and distortioncorrection.

FIGS. 19A-19G show example applications of one or more gabor filters onan image. An image is thresholded by using an adaptive bilateralthreshold on the gabor filter image or by using a morphological filteron the adaptive bilateral threshold image. Using similar techniques tothose described above for image processing, the thresholded imageundergoes erosion and dilation to remove noise. For the secondarycamera, a kernel or structuring element may be used to highlightcircular shapes within the image to locate a contact point within thecircular ring of the gladhand and preserve elliptical/circular objects.

Contours may be located in the threshold image. In an example, thecontours may be used to determine how far the mechand and end effectorare out of alignment with the trailer gladhand. For example, an ellipsemay be fit over each contour, calculating its center and major and minoraxis. Since the gladhand port is approximately a circle, a theoreticalarea may be calculated from the elliptical parameters. The belowequation may be used to calculate the theoretical area of the ellipticalgladhand port approximated as a circle:

${{Theoretical}\mspace{14mu}{Area}} = {\frac{{Major}\mspace{20mu}{Axis}}{2}*\frac{{Minor}\mspace{14mu}{Axis}}{2}*\pi}$

The theoretical area may be cross referenced to the contour's actualarea to compare how well an ellipse approximates the contour. An ellipsemay correspond with, or associated with, the gladhand's pneumatic port.

FIGS. 19A-C show a gabor filter (FIG. 24A) applied to an image to createa texture image (FIG. 19B) that is then thresholded (FIG. 19C), wheremaximum values (shown in black), indicate a change in texture/edges. Thegabor filter of 0 degrees targets edges with vertical orientations.FIGS. 19D-F show a gabor filter (FIG. 19D) applied to an image to createa texture map (FIG. 19E) that is then subjected to a threshold (FIG.19F). The above gabor filter of 90 degrees targets edges with horizontalorientations and produces an edge map seen in FIG. 19F.

The gabor filter may be an orientation-specific filter used for textureanalysis in applications such as edge detection and feature extraction.When used in image processing, the gabor filter returns maximum valuesat locations with texture changes. The parameters can be modified totarget changes in texture along a specific orientation. For example, inFIG. 19A and FIG. 19D, a specified orientation of 0 degrees and 90degrees, respectively, may find vertical edges. As shown in FIG. 19C,the threshold image of the vertical gabor filter shows defined edgeswith vertical orientation. Similarly, in FIG. 19F, the gabor filter of90 degrees may target horizontal edges. In FIG. 19F, now the filter mayidentify edges in the horizontal direction.

These filters can be combined so that a robust edge detection can bemade to find edges of objects of varying size, shape, and surface in alldirections. For example, FIG. 19G shows a combination of several gaborfilters to allow for edge detection to take place along multipledirections, which can help retrieve and edge map. In FIG. 19G, 16different orientations were used to capture the edges in the scene. Thegabor filter may be applied to weight each pixel value depending on thescene's textures/edges. Then the image may be bilaterally blurred, topreserve edges, and threshold.

FIGS. 20A-20C show rectilinear projection and various forms of imagedistortion. Distortion occurs due to the camera lens not being perfectlyaligned. This causes a deviation from perfect rectilinear projection2000A, which is where straight lines appear to be straight in an image.Distortion may morph straight lines to appear to be bent. A perspectivecomparison between distorted and undistorted images is shown in FIGS.20A-20C. As an example, types of distortion include tangentialdistortion 2000B and radial distortion 2000C. Radial distortion 2000Cmay make straight lines appear to buckle the farther away they are fromthe center of the image. Tangential distortion 2000B may make objectsappear nearer than they actually are. The tangential and radialcomponents of lens distortion can be corrected for using what is calledthe Inverse Brown Condray method. The equations to calculate imagedistortion using the Inverse Brown Condray method are as follows:

$x = \frac{u - {pp_{x}}}{f_{x}}$ $x = \frac{v - {pp_{y}}}{f_{y}}$radial  distortion  coefficient = 1 + k₁x²y² + k₂x⁴y⁴ + k₃x⁶y⁶x_(corrected) = (x * radial) + (2p₁xy) + p₂(x²y² + 2x²)y_(corrected) = (y * radial) + (2p₂xy) + p₁(x²y² + 2y²)X = D * x_(corrected)Y = D * y_(corrected) Z = D

Where the distortion coefficients are k₁, k₂, k₃, p₁, p₂, the principalpoint is at pp_(x), pp_(y), and the focal length is f_(x), f_(y). Fromthe camera calibration the focal length, principal point, and distortioncoefficients may be derived. Using the above equations, the worldcoordinates of the point of contact may be calculated. These numbers maybe relayed to the computer and the system may be re-adjusted orre-positioned to better align the end effector with the contact point(e.g., a mating position).

FIG. 21 illustrates an example of a suitable operating environment 2100in which one or more of the present embodiments may be implemented. Thisis only one example of a suitable operating environment and is notintended to suggest any limitation as to the scope of use orfunctionality. Other well-known computing systems, environments, and/orconfigurations that may be suitable for use include, but are not limitedto, personal computers, server computers, hand-held or laptop devices,multiprocessor systems, microprocessor-based systems, programmableconsumer electronics such as smart phones, network PCs, minicomputers,mainframe computers, distributed computing environments that include anyof the above systems or devices, and the like.

In its most basic configuration, operating environment 2100 typicallymay include at least one processing unit 2102 and memory 2104. Dependingon the exact configuration and type of computing device, memory 2104(storing, among other things, APIs, programs, etc. and/or othercomponents or instructions to implement or perform the system andmethods disclosed herein, etc.) may be volatile (such as RAM),non-volatile (such as ROM, flash memory, etc.), or some combination ofthe two. This most basic configuration is illustrated in FIG. 21 bydashed line 2106. Further, operating environment 2100 may also includestorage devices (removable, 2108, and/or non-removable, 2110) including,but not limited to, magnetic or optical disks or tape. Similarly,environment 2100 may also have input device(s) 2114 such as a keyboard,mouse, pen, voice input, etc. and/or output device(s) 2116 such as adisplay, speakers, printer, etc. Also included in the environment may beone or more communication connections, 2112, such as LAN, WAN, point topoint, etc.

Operating environment 2100 may include at least some form of computerreadable media. The computer readable media may be any available mediathat can be accessed by processing unit 2102 or other devices comprisingthe operating environment. For example, the computer readable media mayinclude computer storage media and communication media. The computerstorage media may include volatile and nonvolatile, removable andnon-removable media implemented in any method or technology for storageof information such as computer readable instructions, data structures,program modules or other data. The computer storage media may includeRAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM,digital versatile disks (DVD) or other optical storage, magneticcassettes, magnetic tape, magnetic disk storage or other magneticstorage devices, or any other non-transitory medium which can be used tostore the desired information. The computer storage media may notinclude communication media.

The communication media may embody computer readable instructions, datastructures, program modules, or other data in a modulated data signalsuch as a carrier wave or other transport mechanism and includes anyinformation delivery media. The term “modulated data signal” may mean asignal that has one or more of its characteristics set or changed insuch a manner as to encode information in the signal. For example, thecommunication media may include a wired media such as a wired network ordirect-wired connection, and wireless media such as acoustic, RF,infrared and other wireless media. Combinations of the any of the aboveshould also be included within the scope of computer readable media.

The operating environment 2100 may be one or more computers operating ina networked environment using logical connections to one or more remotecomputers. The remote computer may be a personal computer, a server, arouter, a network PC, a peer device or other common network node, andtypically includes many or all of the elements described above as wellas others not so mentioned. In an example, the operating environment mayinclude one or more vehicle controllers and/or processors associatedwith the vehicle or truck. The logical connections may include anymethod supported by available communications media. Such networkingenvironments are commonplace in offices, enterprise-wide computernetworks, intranets and the Internet.

The different aspects described herein may be employed using software,hardware, or a combination of software and hardware to implement andperform the systems and methods disclosed herein. Although specificdevices have been recited throughout the disclosure as performingspecific functions, one skilled in the art will appreciate that thesedevices are provided for illustrative purposes, and other devices may beemployed to perform the functionality disclosed herein without departingfrom the scope of the disclosure.

As stated above, a number of program modules and data files may bestored in the system memory 2104. While executing on the processing unit2102, program modules (e.g., applications, Input/Output (I/O)management, and other utilities) may perform processes including, butnot limited to, one or more of the stages of the operational methodsdescribed herein.

Furthermore, examples of the invention may be practiced in an electricalcircuit comprising discrete electronic elements, packaged or integratedelectronic chips containing logic gates, a circuit utilizing amicroprocessor, or on a single chip containing electronic elements ormicroprocessors. For example, examples of the invention may be practicedvia a system-on-a-chip (SOC) where each or many of the componentsillustrated may be integrated onto a single integrated circuit. Such anSOC device may include one or more processing units, graphics units,communications units, system virtualization units and variousapplication functionality all of which are integrated (or “burned”) ontothe chip substrate as a single integrated circuit. When operating via anSOC, the functionality described herein may be operated viaapplication-specific logic integrated with other components of theoperating environment 2600 on the single integrated circuit (chip).Examples of the present disclosure may also be practiced using othertechnologies capable of performing logical operations such as, forexample, AND, OR, and NOT, including but not limited to mechanical,optical, fluidic, and quantum technologies. In addition, examples of theinvention may be practiced within a general purpose computer or in anyother circuits or systems.

The embodiments described herein may be employed using software,hardware, or a combination of software and hardware to implement andperform the systems and methods disclosed herein. Although specificdevices have been recited throughout the disclosure as performingspecific functions, one of skill in the art will appreciate that thesedevices are provided for illustrative purposes, and other devices may beemployed to perform the functionality disclosed herein without departingfrom the scope of the disclosure. In addition, some aspects of thepresent disclosure are described above with reference to block diagramsand/or operational illustrations of systems and methods according toaspects of this disclosure. The functions, operations, and/or acts notedin the blocks may occur out of the order that is shown in any respectiveflowchart. For example, two blocks shown in succession may in fact beexecuted or performed substantially concurrently or in reverse order,depending on the functionality and implementation involved.

FIG. 22 depicts an example method 2200 for automated gladhand coupling.The method may be performed using the technology further describedabove. For example, the systems described above may be used toaccomplish method 2200. Method 2200 begins at operation 2202, where atrailer configuration is identified and localized. As further describedherein, one or more cameras may be used to identify a gladhandconnection site on the trailer (e.g., using one or more image processingtechniques and one or more images described above). The real-worldcoordinates of the trailer configuration, such as a gladhand connectionsite or an electrical port, are determined based on the imageprocessing.

At operation 2204, an end effector is positioned. The position of theend effector may be adjusted by using one or more linear actuatorassemblies described herein. The end effector may be positioned based onthe real-world coordinates determined for the identified and localizedtrailer configuration. The position of the end effector may continue tobe adjusted based on feedback from one or more of the image processingtechniques described with respect to operation 2200 and/or other imageprocessing techniques described above. For example, the position of theend effector may be adjusted in a feedback loop based on imageprocessing analysis until image processing determines that the endeffector is positioned properly. For example, the end effector may beadjusted until the end effector is in a mating position (e.g., aposition at which a sealing surface of a gladhand clamped by the endeffector aligns with a sealing surface of a gladhand connection site).The mating position may also consider a rotational orientation of theend effector and/or clamped gladhand relative to the mating surface ofthe gladhand connection site. For example, the flange(s) of the clampedgladhand may be intentionally misaligned with flange(s) of the gladhandconnection site such that rotation of the gladhand will cause couplingof the gladhand to the gladhand connection site.

At operation 2206, a gladhand is caused to be rotated by the endeffector. After positioning the end effector (and gladhand clamped bythe end effector) in a mating position, the end effector may rotate thegladhand to couple the clamped gladhand with the gladhand connectionsite at the current position. The rotation of the gladhand may result inpressurized coupling of the gladhand at the gladhand connection site onthe trailer.

At operation 2208, the end effector is repositioned. After rotating thegladhand to couple the gladhand at the gladhand connection site, the endeffector may release the clamped gladhand by transitioning to anunclamped configuration. The unclamped configuration may increase adistance between a top clamp and a bottom clamp of the end effector torelease (e.g., decouple) the gladhand. The end effector may berepositioned (e.g., rotated or moved about three-dimensional space) by amotor of the end effector and/or by a linear actuator assembly. In anexample, after the gladhand is coupled to the trailer, the end effectormay be moved to a resting position, such as a position with presetcoordinates of the end effector. The repositioning of the end effectormay not use image processing techniques. Decoupling of the gladhand fromthe trailer may perform similar steps as coupling the gladhand to thetrailer. For example, a gladhand coupled to a trailer may be identifiedand localized. The end effector may then be positioned based on theidentified and localized gladhand. For example, the end effector may bemoved (e.g., by a linear actuator assembly) and/or oriented so that theend effector can couple to the gladhand. The end effector may thenrotate the gladhand to decouple the gladhand from the trailer. Imageprocessing techniques described herein may be used to determine at whatposition and orientation of the end effector that the gladhand is nolonger coupled to the trailer. After decoupling the gladhand, the endeffector may be repositioned (e.g., while clamping the gladhand). In anexample, the end effector may perform operations similar to thosedescribed above to couple the gladhand to the truck, rather than thetrailer (e.g., if the trailer has reached its destination).

This disclosure describes some embodiments of the present technologywith reference to the accompanying drawings, in which only some of thepossible embodiments were shown. Other aspects may, however, be embodiedin many different forms and should not be construed as limited to theembodiments set forth herein. Rather, these embodiments were provided sothat this disclosure was thorough and complete and fully conveyed thescope of the possible embodiments to those skilled in the art.

Further, as used herein and in the claims, the phrase “at least one ofelement A, element B, or element C” is intended to convey any of:element A, element B, element C, elements A and B, elements A and C,elements B and C, and elements A, B, and C. In addition, one havingskill in the art will understand the degree to which terms such as“about” or “substantially” convey in light of the measurementstechniques utilized herein. To the extent such terms may not be clearlydefined or understood by one having skill in the art, the term “about”shall mean plus or minus ten percent.

Although specific embodiments are described herein, the scope of thetechnology is not limited to those specific embodiments. One skilled inthe art will recognize other embodiments or improvements that are withinthe scope and spirit of the present technology. In addition, one havingskill in the art will recognize that the various examples andembodiments described herein may be combined with one another.Therefore, the specific structure, acts, or media are disclosed only asillustrative embodiments. The scope of the technology is defined by thefollowing claims and any equivalents therein.

What is claimed is:
 1. A mechanized gladhand comprising: a sealingsurface; a collar; a plunger; a turret rotatable about the collar andthe plunger; and a retention spring applying a threshold force on theplunger and the collar to limit rotation relative to each other.
 2. Themechanized gladhand of claim 1, the mechanized gladhand furthercomprising: a connector plate; and a detent plate.
 3. The mechanizedgladhand of claim 2, wherein the connector plate and the detent plateare coupled to the turret to rotate with the turret.
 4. The mechanizedgladhand of claim 1, wherein the turret is coupled to a hose and whereinthe hose is coupled to a truck.
 5. The mechanized gladhand of claim 1,wherein the hose is fluidly coupled to a duct in the plunger and a portin the sealing surface.
 6. The mechanized gladhand of claim 5, whereinthe duct in the plunger and the port in the sealing surface areconcentric.
 7. The mechanized gladhand of claim 1, wherein the collarincludes a connector plate and a detent plate.
 8. The mechanizedgladhand of claim 1, wherein the plunger is retained to the turret on arotational bearing.
 9. A method for automating a gladhand couplingbetween a vehicle and a trailer, the method comprising: identifying, bya processor, a trailer mating surface, based on at least one image;determining a mating position of the trailer mating surface, based onthe at least one image; positioning an end effector based on the matingposition, such that a gladhand mating surface of a gladhand coupled tothe end effector is coupled to the trailer mating surface; rotating theend effector and the gladhand relative to the trailer mating surface andthe gladhand mating surface, at the mating position; decoupling thegladhand from the end effector by opening a clamp of the end effector;and repositioning the end effector.
 10. The method of claim 9, whereinthe at least one image includes a first image and wherein positioningthe end effector is further based on a second image.
 11. The method ofclaim 10, wherein the first image is obtained from a first camera andthe second image is obtained from a second camera.
 12. The method ofclaim 11, wherein the second camera is coupled to end effector.
 13. Themethod of claim 9, wherein positioning the end effector includescontrolling at least one linear actuator coupled to the end effector.14. The method of claim 9, wherein repositioning the end effectorincludes controlling at least one linear actuator coupled to the endeffector.
 15. The method of claim 9, wherein opening the clamp of theend effector includes moving a traveler along a drive shaft of the endeffector.
 16. A method for automating a gladhand decoupling between avehicle and a trailer, the method comprising: identifying, by aprocessor, a gladhand with a gladhand mating surface coupled to atrailer mating surface, based on at least one image; determining amating position of a gladhand coupled to the trailer mating surface,based on the at least one image; positioning an end effector based onthe mating position, such that the end effector becomes coupled to thegladhand; rotating the end effector and the gladhand relative to thetrailer mating surface and the gladhand mating surface, at the matingposition; and repositioning the end effector and the gladhand.
 17. Themethod of claim 16, wherein determining the mating position is furtherbased on machine learning.
 18. The method of claim 16, wherein the atleast one image includes a first image and wherein positioning the endeffector is further based on a second image.
 19. The method of claim 16,wherein repositioning the end effector and gladhand includes couplingthe gladhand to the vehicle.
 20. The method of claim 16, whereinrotating the end effector and the gladhand relative to the trailermating surface decouples the gladhand mating surface from the trailermating surface.